Skip to content

Feat: custom audio backend (FFmpeg + miniaudio) to replace libmpv#894

Draft
dweymouth wants to merge 13 commits intomainfrom
feat/mpv-replacement
Draft

Feat: custom audio backend (FFmpeg + miniaudio) to replace libmpv#894
dweymouth wants to merge 13 commits intomainfrom
feat/mpv-replacement

Conversation

@dweymouth
Copy link
Copy Markdown
Owner

@dweymouth dweymouth commented Apr 1, 2026

Summary

Replaces libmpv as the local audio engine with a bespoke stack built on FFmpeg (libav*) and miniaudio. Gated behind the localav build tag so mpv remains the default until this is production-ready.

Implements #360

Build:

go build -tags "migrated_fynedo localav" ./...

Why

The primary motivation is direct PCM sample access to enable additional visualizers such as ProjectM — libmpv's abstraction layer prevents tapping decoded frames — as well as other new audio features such as crossfade playback that aren't supported by MPV. A lower-level pipeline also removes the large libmpv native binary dependency.

Stack

Layer Library
Demux / HTTP streaming libavformat
Decode libavcodec
EQ / ReplayGain / metering libavfilter
Audio output + device enum miniaudio 0.11.21 (vendored single-header)

Features

  • Gapless playback — next decoder is pre-opened and swapped in before the ring buffer drains, so there is no audible gap at track boundaries
  • SPSC lock-free ring buffer between the decode goroutine and the miniaudio callback
  • 15-band parametric EQ via avfilter (same filter-string format as mpv backend)
  • ReplayGain from file tags via avfilter volume node
  • Audio device enumeration and exclusive mode (CoreAudio / WASAPI / ALSA)
  • Peak/RMS metering via astats avfilter
  • Seek safety — decode goroutine stopped synchronously before filter-graph rebuild
  • PCM sample hook point ready for a future ProjectM visualizer

Commit structure

  1. Refactor — promote shared Equalizer, AudioDevice, MediaInfo, and LocalPlayer interface out of the mpv package into backend/player, so both backends can satisfy the same contract without importing each other
  2. feat — the new backend/player/localav/ backend + build-tagged app-init files

What's not done yet

  • ICY radio metadata (stub no-ops; mpv backend still handles radio)
  • Windows / Linux build validation (only tested on macOS + Homebrew FFmpeg)
  • Flatpak / AppImage packaging adjustments

Test plan

  • go build -tags "migrated_fynedo localav" ./... compiles cleanly
  • go test -tags "migrated_fynedo localav" ./backend/player/localav/... passes
  • Play FLAC, MP3, AAC, Opus, OGG — confirm audio output
  • Gapless transition between tracks — no audible gap
  • Seek mid-track — position jumps correctly, no desync
  • Toggle EQ bands — audio changes without crash
  • Set ReplayGain to track mode — loudness difference audible
  • Open audio device selector — devices list, switching works
  • Enable peak meter in Now Playing — values update during playback
  • Default mpv build (go build -tags migrated_fynedo ./...) still compiles and plays

🤖 Generated with Claude Code

@dweymouth dweymouth changed the title feat: localav audio backend (FFmpeg + miniaudio) Feat: custom audio backend (FFmpeg + miniaudio) to replace libmpv Apr 4, 2026
dweymouth and others added 11 commits April 5, 2026 17:22
Define LocalPlayer interface and shared AudioDevice/MediaInfo types in
backend/player so both mpv and a future alternative backend can implement
the same contract without importing each other.

- backend/player/equalizer.go: Equalizer interface + ISO10/15BandEqualizer
  moved from backend/player/mpv; mpv/equalizer.go is now type aliases
- backend/player/localplayer.go: LocalPlayer interface, AudioDevice, MediaInfo
- backend/player/mpv/player.go: implements player.LocalPlayer
- backend/app.go: LocalPlayer field is now player.LocalPlayer interface;
  initMPV/setupMPV renamed to initLocalPlayer/setupLocalPlayer (dispatch
  via build-tagged files)
- backend/app_player_mpv.go: //go:build !localav — extracts initLocalPlayer
  for the mpv backend
- backend/playbackengine.go, ui/**:  update type assertions and imports to
  use player.* types instead of mpv.*

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New alternative local player backend gated behind the `localav` build tag.
Build with: go build -tags "migrated_fynedo localav" ./...

Stack:
- libavformat/libavcodec/libavfilter for demux, decode, and DSP
- miniaudio (vendored single-header 0.11.21) for audio output and
  device enumeration via CoreAudio/WASAPI/ALSA

Key features:
- Gapless playback via double-slot decoder: next track is pre-opened and
  swapped in immediately when the current decoder hits EOF, before the
  ring buffer drains, eliminating audible gaps
- SPSC lock-free ring buffer between decode goroutine and miniaudio callback
- 15-band parametric EQ via avfilter graph (same filter string as mpv backend)
- ReplayGain from file tags via avfilter volume node
- Audio device enumeration and selection including exclusive mode
- Peak/RMS metering via astats avfilter
- Seek safety: decode goroutine is stopped synchronously before
  av_player_seek rebuilds the filter graph (prevents SIGSEGV race)
- Direct PCM sample access hook point for future ProjectM visualizer

Files:
- backend/player/localav/av_player.{h,c} — C engine
- backend/player/localav/player.go — CGo bindings, implements LocalPlayer
- backend/player/localav/miniaudio.h — vendored miniaudio 0.11.21
- backend/player/localav/cgo_{darwin,linux,windows}.go — platform CGo flags
- backend/app_player_localav.go — //go:build localav app init

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace decode-ahead time tracking with frame-counter position formula
(position_offset + (frames_played_total - position_clock_ref) / sample_rate)
so the UI position reflects actual audio output, not the 4-second decode
lookahead in the ring buffer.

Move peak meter computation from the astats AVFilter (decode-ahead path)
into the miniaudio callback (actual playback path) so peaks are in sync
with audible audio. This also eliminates the SIGSEGV-prone filter graph
rebuild that av_player_set_peaks_enabled previously triggered.

Fix double mutex unlock in gapless swap (decoder_lock released inside the
if-block then again unconditionally), which is UB on non-recursive
pthread_mutex and could cause the decode loop to exit permanently.

Fix gapless track-change threshold to use frames_played_total + ring.fill
instead of frames_written_total, which silently drifts after ring_clear
in av_player_seek.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
EQ bands and preamp are now applied via ma_peak2 biquad filters in the
miniaudio callback rather than FFmpeg's filter graph. This gives immediate
EQ response without waiting for the ~4s ring buffer to drain, and ensures
the peak meter reflects raw pre-EQ decoded audio as intended.

Go now passes numeric band params (frequency, gain_db, Q) to C directly
instead of building FFmpeg avfilter strings. A double-buffered eq_bank_t
with an atomic index swap keeps the audio callback lock-free.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Use a frame-position-keyed bitrate history ring to delay the displayed
bitrate by the ring buffer depth (~4s), so the UI shows the bitrate
matching the audio currently playing rather than the most recently
decoded packet. Also fixes post-seek desync by realigning
frames_written_total to frames_played_total on seek.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move ReplayGain from FFmpeg filter graph (decode stage) to the miniaudio
output callback, matching the EQ pattern. The decoder reads all four RG
tags (TRACK/ALBUM GAIN/PEAK) from file metadata at open time. A linear
gain multiplier is computed from the mode, prevent-clip flag, and preamp
offset, then applied in the callback after EQ and before master volume.
For gapless playback, a pending gain switch is armed at the track
boundary so the new track's gain takes effect exactly when its audio
starts playing through the ring buffer.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
av_player_open and av_player_stop freed next_dec without holding
decoder_lock, racing with av_player_open_next which acquires the lock.
When the time-pos polling goroutine called SetNextFile concurrently with
a user-initiated PlayFile, both could free the same decoder.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@dweymouth dweymouth force-pushed the feat/mpv-replacement branch from 750fa75 to 63787d7 Compare April 6, 2026 00:22
dweymouth and others added 2 commits April 5, 2026 17:56
Add av_waveform.c implementing av_analyze_waveform() directly via FFmpeg,
replacing the MPV transcode-to-WAV pipeline. Split StartWaveformGeneration
into build-tag-gated files (waveformimage_localav.go / waveformimage_mpv.go)
so both backends are fully supported. Analysis supports cooperative
cancellation via an atomic cancel flag, wired to the job context so
seeking/navigation stops in-flight decodes promptly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@ocelotsloth
Copy link
Copy Markdown
Contributor

ocelotsloth commented Apr 9, 2026

#720 (comment)

It's reached feature parity with the MPV backend now, except for some issues with exclusive mode at least on Mac, so I think it proves the libav* and Miniaudio stack is viable, and I'll probably stick with it

Well it certainly works on Linux. I did have to update the unit tests a bit to generate some pink noise instead of reading MacOS system files to get some synthetic audio.

I also made some changes to enable headless mode so that we can use libav and other player implementations can swap in for Miniaudio.

I'll submit a PR to your development branch tomorrow night with just those changes so you can get a look at it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants